SAN Fabric Design Rules 




The sections in this chapter contain SAN fabric design configuration rules for heterogeneous 
SANs implemented using the HP B-Series product line of Core, Director, and SAN Switch 
Fibre Channel switch models and the MP Router, the HP C-Series product line of Cisco 
Director and Fabric Fibre Channel switch models resold by HP, and the HP M-Series product 
line of Director and Edge Fibre Channel switches 

Support is provided for SAN fabrics consisting of the Fibre Channel switches listed in 
Table 18, HP StorageWorks B-Series Product Line Switches, the switches listed in Table 21, 
HP C-Series Product Line Switches, or the switches listed in Table 24, HP StorageWorks 
M-Series Product Line Switches. 

For new SAN deployments HP recommends you utilize switch models from a single product 
line exclusively. To meet the needs of customers desiring a mix of switch models from the 
different product lines however, HP also supports two levels of SAN fabric interoperability. 
Specifically, HP supports: 

■ Within a multi-fabric SAN, one fabric with all B-Series switches and another fabric with 
all M-Series switches. This is referred to as a "dual heterogeneous SAN fabric" 

■ Within a single fabric, switch models from different product lines intermixed. This is 
referred to as an "interoperable heterogeneous SAN fabric". For this level of 
interoperability, three combinations are supported: 

— B-Series SAN switch models with M-Series Director and Edge switch models 

— B-Series switch models with C-Series switch models 

— C-Series switch models with M-Series switch models 

Refer to Heterogeneous/Interoperable SAN Fabrics, page 98, for the fabric interoperability 
rules and configuration rules for these two levels of interoperability. 

The configuration rules for a SAN begin with the SAN fabric rules. These are then modified 
depending on the specific topology implementation rules, platform or operating system and 
storage system rules described here and in Chapter 4, "Heterogeneous SAN Platform and 
Storage System Rules", and the requirements of applications being run on the SAN. Read the 
documentation and release notes for all hardware and software products that are being utilized 
in the SAN for additional configuration information details. See the Preface, "Related 
Documents" for a list of related documentation. 



The Storage Networking Industry Association (SNIA) sponsors the Supported Solutions 
Forum (SSF). SSF is a storage industry program that improves the interoperability of Fibre 
Channel products. HP is a member of SNIA and SSF. 
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HP supports a number of Fibre Channel switch models in configurations that are compliant 
with the guidelines defined by the SNIA Supported Solutions Forum. SSF has defined 
numerous configurations with heterogeneous servers and heterogeneous storage systems. 

For additional information on this program, refer to http:/ / www.snia.org/ ssf 

Supported HP StorageWorks Router Models 

HP supports four router products/functionalities providing multiple routing and SAN 
extension capabilities. Table 17 lists the currently supported HP router products along with the 
supported protocols and supported Fibre Channel switch product lines. Refer to Chapter 7, 
"SAN Extension" for FCIP and iSCSI configuration rules. Additionally, for information on HP 
support of certain third party SAN Extension products, refer to the Continuous Access and 
Data Replication Manager SAN Extensions Reference Guide: 

http://h200002.www2.hp.com/bc/docs/support/SupportManual/c00062684/c00062684.pclf 



Table 17: HP StorageWorks Router Products 







r 


Number of Ports 


Supported 
Protocol(s) 


Supported 
Switch Product 
Line(s) 


HP StorageWorks 
Multi-Protocol Router 


7.1.2c 


8/16 

Refer to Table 19 


Fibre Channel 
Routing, FCIP 


B-Series 


C-Series Inter-VSAN 
Routing 


1.34a 


Refer to Table 22 


Fibre Channel 
Routing 


C-Series 


4/8-Port IP Storage 
Services module 


2.0.1b 


4/8 


iSCSI, FCIP 


C-Series 


Multiprotocol Service 
module 


2.0.1b 


2 


iSCSI, FCIP 


C-Series 


HP StorageWorks 
SR2 122-2 IP Storage 
Router 


3.4.5 


2 Fibre Channel/2 
GigE 


iSCSI, FCIP 


B-Series, 
C-Series, 
M-Series 


HP StorageWorks 
SR2122 iSCSI Storage 
Router 


3.1.1 


2 Fibre Channel/2 
GigE 


iSCSI 


B-Series, 
C-Series, 
M-Series 



Supported Switch Models - B-Series Product Line 

HP supports 1 Gbps, 2 Gbps, and 4 Gbps SAN B-Series product line Fibre Channel switch 
models. These switch models include legacy products released by HP and Compaq. The 
relationship between the HP and Compaq switch products is shown in Table 18. Refer to the 
section, SAN Fabric Rules - B-Series Product Line, for specific switch model support rules. 



Note: Refer to the following Web page for the latest information on supported B-Series Fibre 
Channel switch models ana firmware versions: 

http://hl 8006. wwwl .hp.com/storage/saninfrastructure.html 
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Table 18: HP StorageWorks B-Series Product Line Switches 



HP StorageWorks Switch Name 


Firmware 
Version 


i 1 


HP StorageWorks MSA SAN switch 2/8 




8 


HP StorageWorks SAN Switch 2/8 EL, 2/8 power pack 


3.2.0 


8 


HP StorageWorks SAN Switch 2/16, 2/16 EL, 2/16 power pack 




16 


HP StorageWorks SAN Switch 2/8V TAA, 2/8V TAA power pack 




8 


HP StorageWorks SAN Switch 2/ 16V, 2/1 6V TAA, 2/1 6N FF, 
2/1 6N power pack, 2/1 6N FF TAA, 2/1 6N TAA power pack 




16 


HP StorageWorks SAN Switch 2/32, 2/32 power pack 




32 


HP StorageWorks Core Switch 2/64, 2/64 power pack 


4.4.0 


64 (2 switches per chassis, for a 
total of 1 28 ports per chassis) 


HP StorageWorks SAN Director 2/128, 2/128 power pack 




128 


HP StorageWorks SAN Switch 4/32 base/full 




16/32 


HP StorageWorks SAN Switch 4/32 power pack 




32 


HP Switch Name 


Compaq StorageWorks 
Switch Name 




Number of Ports 


HP Brocade 2400 
(HP reseller) 


Compaq StorageWorks 
SAN Switch 8 




8 


N/A 


Compaq StorageWorks 
SAN Switch 8-EL 




8 


HP Brocade 2800 
(HP reseller) 


Compaq StorageWorks 
SAN Switch 16 




16 


N/A 


Compaq StorageWorks 
SAN Switch 16-EL 


2.6.2b 


16 


HP Surestore FC Switch 6164 
(64 ISL Ports) 


Compaq StorageWorks 
SAN Switch Integrated/32 
(64 ISL Ports) 




32 (counts as 6 switches and 2 hops 
when applying configuration rules) 


HP Surestore FC Switch 6164 
(32 ISL Ports) 


Compaq StorageWorks 
SAN Switch Integrated/64 
(32 ISL Ports) 




64 (counts as 6 switches and 2 hops 
when applying configuration rules) 


HP Surestore FC lGb/2Gb Entry 
Switch 8B 


N/A 




8 


N/A 


Compaq StorageWorks 
SAN Switch 2/8-EL 




8 


N/A 


Compaq StorageWorks 
SAN Switch 2/16-EL 


3.2.0 


16 


HP Surestore FC lGb/2Gb 
Switch 8B 


N/A 




8 


HP Surestore FC lGb/2Gb 
Switch 16B 


Compaq StorageWorks 
SAN Switch 2/16 




16 
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SAN Fabric Rules - B-Series Product Line 

All switch models shown in Table 18 are supported in the HP StorageWorks SAN provided 
that the same firmware versions and switch settings for each switch model family are utilized 
for the corresponding models listed, and specific switch model configuration rules are 
followed. For additional information on operating system HBA/driver/firmware/software 
support, contact your HP field representative. 

■ For SAN fabrics consisting exclusively of Compaq switch models or a mix of pre-merger 
HP and pre-merger Compaq switch models (Compaq StorageWorks switch name, HP 
switch name, or Brocade switch name sold by HP in the above table), use the Compaq 
default switch settings. Configuration files with these settings are available from HP 
Services. 

■ For SAN fabrics consisting exclusively of pre-merger HP switch models (HP switch name 
or Brocade switch name sold by HP in the above table), utilize pre-merger HP switch 
settings. Configuration files with these settings are available from HP Services. 



Note: The HP StorageWorks switches listed utilize common settings by default. 



Fabric and Switch Model Maximums - B-Series Product Line 

The fabric maximums listed are for B-Series switch model SAN fabrics utilizing HP 
XP128/1024, XP48/512, XP256, VA7100, VA7110, VA7400, VA7410, Enterprise Virtual 
Array (EVA), EMA/ESA12000, EMA16000, MA/RA8000, MA6000, MSA1000, MSA1500, 
RA4000, or RA4100 storage systems. Refer to Chapter 4 for specific operating system support 
for each storage system type based on the switch product line used. 

In general, these fabric rules also apply to Continuous Access XP, EVA, and DRM for 
EMA/ESA12000, EMA 16000, and MA/RA8000. Refer to Chapter 4 of this guide and the 
Continuous Access EVA Design Reference Guide for additional details. 

The following rules are for SAN fabrics implemented with the switch FW versions listed in 
Table 18, unless otherwise stated. 

1. Up to 34 switches and up to 1280 total ports in a single SAN fabric. Each fabric may 
contain any combination of supported 1 Gbps and 2 Gbps switch models listed, provided 
the individual switch model fabric limits listed below are not exceeded. 

2. HP StorageWorks SAN Switch 4/32 - 4 Gbps port speed supported only for ISL 
connectivity between 4/32 switches. 

3. The HP StorageWorks SAN Director 2/128 - maximum of 10 chassis total per fabric, each 
chassis adds 1 to the fabric switch count. 

4. The HP StorageWorks Core Switch 2/64 - maximum of 10 chassis total per fabric, each 
chassis contains 2 logical switches (See Figure 33), thus 10 chassis add 20 to the fabric 
switch count. 

The Core Port Identifier (PID) addressing mode is required on all other switches in the 
same fabric with the HP StorageWorks SAN Switch 2/32, Core Switch 2/64, and SAN 
Director 2/128. Refer to SAN Core and SAN Switch Addressing Mode. 

5. For SAN fabrics containing any 1 Gbps switch models utilizing 2.6.x or later firmware, 
the maximum number of user ports supported is 728. With Security enabled, the 
maximum number of user ports supported is 500. 
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For SAN fabrics containing exclusively 2 Gbps switch models, the maximum number of 
user ports supported is 1280. With Security enabled, the maximum number of user ports 
supported is 728. 

6. For SAN fabrics with Security enabled, the Security database sizes are limited as follows. 

— In a mixed fabric with 1 and 2 Gbps switches, the maximum size of the security 
database is 32 KB, with only 16 KB active. 

— In a 2 Gbps only fabric, the security database size can be 128 KB, with 64 KB active. 

— For all 1 Gbps/2Gbps mixed fabrics, the maximum number of Device Connection 
Control (DCC) policies is limited to 620. 

7. Zoning database size limits - For SAN fabrics containing 1 Gbps switch models utilizing 
2.6. lx or later firmware, or 2 Gbps switch models utilizing 3. lx or later firmware, the 
zoning database size must not exceed 96 KB. 

For SAN fabrics containing 2 Gbps switch models utilizing 4.1x or later firmware, the 
zoning database size must not exceed 128 KB. 



Note: Use the "cfgSize" command to determine the size of the Zoning database. 



8. StorageWorks SAN Switch Integrated 32 or 64, HP Surestore FC Switch 6164 - 
maximum of 4 chassis total per fabric, each chassis adds 6 switches to the fabric switch 
count. Maximum fabric configuration is 4 chassis with 4 other SAN switch model 
switches. 

9. Up to 7 switch hops (8 switches) maximum between any two devices in a SAN fabric. 
Each SAN Switch Integrated 32 or 64 or HP Surestore FC Switch 6164 model switch 
utilized in a fabric adds up to 2 hops to the hop count between devices depending on the 
specific device-to-switch connections and device-to-device access, (refer to Chapter 2, 
Figure 6 and Figure 7). 

10. StorageWorks SAN Switch 2/8-EL - By default, this switch is supported in SAN fabrics 
with up to 4 switches total only. A license upgrade is available to allow these switches to 
be upgraded for support in larger fabrics with greater than 4 switches total. 

1 1. HP Surestore FC lGb/2Gb Entry Switch 8B - Supported with a single E-port connection. 

12. Compaq Fibre Channel Storage Switch 8 or Fibre Channel Storage Switch 16 models- up 
to 4 switches total per fabric using these model switches only, or when intermixed with 1 
Gbps SAN switches. 



Note: Intermixing of Compaq Fibre Channel Storage Switch 8 and Fibre Channel Storage Switch 16 
models and 1 Gbps StorageWorks SAN Switch models requires compatibility mode (VC Encoded 
Address Mode) be set in the SAN Switches (refer to the SAN Switch documentation). Intermixing of 1 
Gbps Compaq Fibre Channel Storage Switch 8 or Fibre Channel Storage Switch 16 switch models 
and 2 Gbps switch models is not supported 



13. The Compaq FC-AL Switch 8 is supported for cascaded attachment to the SAN through a 
single FL-port on a Compaq SAN Switch 8, SAN Switch 16, SAN Switch 8-EL, or SAN 
Switch 16-EL. In this configuration, RA4000/4100 storage systems are accessible only 
from servers attached directly to the FC-AL switch. 
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Note: Cascaded attachment of the FC-AL Switch 8 connected to 2 Gbps switch models is not 
supported 



14. Within a single fabric where switches are interconnected, each switch must have a unique 
domain number (Domain ID) and a unique World Wide Name (WWN). All switch 
configuration parameters in each switch must be the same. 



Note: Do not configure any switches with a domain ID of 8. HP-UX systems reserve domain 8 for 
Private Loop devices. 



15. 



16. 



17. 



18. 



Optional switch features may be used on any switch in the fabric if the feature is supported 
on that switch. This includes features such as ISL Trunking, QuickLoop, Fabric Watch, 
Advanced Performance Monitoring, and Extended Fabrics. Refer to the switch model 
specific documentation to determine which features are available for a given switch 
model. 

Any mix of servers and storage systems is allowed in a SAN provided the specific 
platform, operating system, and storage system fabric limits and rules are followed. Refer 
to the appropriate sections in this guide and the documentation listed in the section 
"Related Documents" in the preface. 

HP requires that all switches in a single fabric or multi-fabric SAN use the same switch 
firmware revision for each switch model family. Two successive fabric firmware versions 
can be temporarily used in one fabric or multiple fabrics in a SAN during switch firmware 
rolling upgrades. 

For B-Series product line, Table 18 Fibre Channel switches - Up to 15 switches 
configured in a single ring with a Ring SAN fabric topology and no cascaded switches. Up 
to 22 switches in a Ring SAN fabric with cascaded switches, provided that no more than 
1 1 switches are in a ring and no more then 1 1 switches are outside of the ring. One outside 
switch is cascaded from each of the 1 1 ring switches. 



TERMINOLOGY: 
One 2/64 Core Switch 

"Chassis Box"= 2 
logical 64-port switches 




Figure 33: Core Switch definition 
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A maximum of 34 switches 




A maximum of 34 switches 



8/16/32- 



8/16/32 





64-port 


64-pc 


>rt 



64-port 



64-port 



8/16/32 



8/16/32 



HA no single 
point of 

failure (NSPOF) 
support 
requires a 

duplicate SAN 
fabric 




Figure 34: Two Fabrics for no single point of failure high availability 



See also Chapter 2, Figure 20. 



Fabric and MP Router Maximums - B-Series Product Line 

The MP Router maximums listed in this section describe the rules and limits when using the 
MP router for Fibre Channel routing in SAN fabrics consisting of any of the B-Series switch 
models listed in Table 18. The rules apply for fabrics utilizing HP XP128/1024, XP48/512, 
VA7100, VA7110, VA7400, VA7410, Enterprise Virtual Array (EVA), EMA/ESA12000, 
EMA16000, MA/RA8000, MA6000, MSA1000, or MSA1500 storage systems. Refer to 
Chapter 4 for specific operating system support for each storage system type based on the 
switch product line used. 

Table 19 lists the scalability rules for Meta SANs utilizing the MP Router. Refer to Figure 36 
and Figure 36 for additional descriptions of terminology used in the rules listed in the table. 



Table 19: HP StorageWorks MP Router Scalability Rules 



Edge Fabric Scalability 


Domains 
(see Figure 36) 


Maximum number of front domains (# EX-ports) per edge fabric 


24 


Maximum number of translate domains (# of remote fabrics) per edge fabric 


6 


Maximum number of physical switches in an edge fabric 


34 


Maximum number of total domains per edge fabric (physical domains + front domains + 
translate domains) 


64 


Devices 
(User Ports) 


Maximum number of local and remote devices per edge fabric (For edge fabrics with 
greater than 600 devices, v4.2.0c or later is required for all switches in the fabric when 
using the MP Router 


1280 


Maximum number of imported devices per edge fabric 


1000 






Zoning 


Maximum number of entries per LSAN zone 


200 


Routed Fabrics (Meta SAN) Scalability 


Edge Fabrics 


Maximum number of edge fabrics connected to a routed fabric 


32 


Meta SAN 


Maximum number of total ports per routed fabric 


4000 


Routers 


Maximum number of MP Routers per routed fabric 


3 



SAN Design Reference Guide 



89 



SAN Fabric Design Rules 



Table 19: HP StorageWorks MP Router Scalability Rules 



LSAN 


Maximum number of LSAN device entries per routed fabric 


3000 


Maximum number of LSAN zones per routed fabric 


1000 


Hop Count Scalability 


Hop Count 


Maximum number of hops between switches (including routers) in a Meta SAN. See 
Figure 35 - diagram of hop counts 


7 




Fabric A 



Sarilch i 

Switch 7 



Switch 3 I i 

Switch 4 I | 

Jrrr MPR 1 [-1 

' Switch 5 



Switch 

Switch 7 




Fabric B 



7 hops using 1 MP Router 



Figure 35: Maximum hop count 




(EX_ports) 



Fabric A 

Up to 6 
Translate 
Domains 

#1.. 



Each Fabric can have up to 24 Front 
Domains (24 EX-ports) 

Figure 36: Front and translate domains 
Additional MP Router Configuration Rules 



Fabric B.. 

Translate Domain 
#1... 




MP Router 



MP Router 



Each Fabric can have access to up to 6 
translate domains (6 remote fabrics) 













L__^\ 

N I ' ■ 


MP Router 





...Fabric G 

...Translate 
Domain #6 



1. All configurations must use the default settings for R_A_TOV (10000 seconds) and 
E_D_TOV (2000 seconds). 

2. Direct connectivity of devices to MP Router ports is not supported. All devices in a Meta 
SAN must be connected to Fibre Channel switch ports. 
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3. SAN boot through the MP Router is not currently supported. 

4. For redundancy, HP recommends that a minimum of 2 EX-Ports (MP Router ISLs) be 
used for each fabric connected to the MP Router. The number of EX-Ports required is 
based on the performance requirements for all devices being shared between fabrics 
through the MP Router. 

■ For initial settings, calculate the number of EX-Ports using a 5 : 1 ratio of standard ISLs 
or switch device ports to MP Router EX-Port ISLs. Monitor port performance to 
determine if this ratio is acceptable based on usage (use the "portperf show" 
command to monitor MP Router port performance). 

■ For very high bandwidth requirements and applications, use a ratio of 3 : 1 or 1:1. 

MP Router XPath OS and Fibre Channel Switch and Fabric OS Compatibility 

Table 20 summarizes the B-Series Fibre Channel switch models and firmware versions that are 
supported with the MP Router XPath OS v7.1.2c. 



Table 20: XPath OS v7.1 .2 Supported B-Series Switches and Firmware 







1 Gb 


2 Gb 2/8, 2/8-EL, 
2/16, 
2/16-EL, 
Surestore 8B, 1 6B 


2 Gb 2/8V, 
2/16V,2/16N, 
2/32, 2/64, 
2/128 




Fabric 
Manager 1 


HP Minimum 
Versions 


v2.6.2a 


v3.1.2a 


v4.2.2a 


v4.4.0 


v4.4 


HP 

Recommended 
Versions 


v2.6.2b 


v3.2.0 


v4.4.0 


v4.4.0 


v4.4 



1 . The use of Fabric Manager is strongly recommended in all configurations utilizing the MP Router 



SAN Core and SAN Switch Addressing Mode 

When using products from the B -series product line, two different addressing modes are 
available. As SAN configurations grow to include more switches, HP recommends that the 
"Core Switch" addressing mode should be used. This is obtained by setting the Core Switch 
PID configuration parameter bit. 

Certain switches previously supplied by HP and Compaq were shipped with the Core Switch 
PID configuration bit cleared. When the switches are operated in this mode, certain restrictions 
apply regarding the maximum number of switches in a fabric and the maximum number of 
ports on a switch. If a StorageWorks Core Switch 2/64, SAN Director 2/128, or StorageWorks 
SAN switch 2/32 is used anywhere in a fabric, then all the switches in the fabric must have the 
Core Switch PID configuration bit set. Currently, all switches are shipped with the Core 
Switch PID bit set. 



Note: The MP Router supports using different PID formats on different fabrics and routing frames 
between them. 



SAN managers with existing fabrics that have switches with the Core PID bit cleared must 
decide whether to set the Core Switch PID bit now or later. The trade-offs are as follows: 
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■ All switches in a fabric must have the same Core Switch PID bit setting, whether it is set 
or cleared, otherwise the fabric will segment. It must be set on all switches if a 
StorageWorks SAN switch 4/32, StorageWorks Core Switch 2/64, SAN Director 2/128, or 
StorageWorks SAN switch 2/32 is part of the fabric. 

■ With two fabrics, the Core Switch PID can be changed on one fabric at a time, allowing 
the SAN storage system to operate during the changeover. 

■ HP-UX and IBM AIX systems use the address bits to identify logical units, when the 
addressing bits are changed, the logical unit definitions must also be changed. This 
requires a reboot of the servers and cannot be done without taking down the entire SAN 
storage system in a planned maintenance scenario. 

■ If the existing switches in a fabric have the Core PID bit cleared, this bit will need to be 
cleared on any new switches added, since all switches ship with the Core PID bit set. 

HP recommends that this change be performed to avoid potential problems in the future. 
Additional information on the Core Switch PID is available at: 

http://hl 8006. wwwl .hp.com/storage/saninfrastructure.html 

Supported Switch Models - C-Series Product Line 

HP currently supports six models of the C-Series product line 2 Gbps Fibre Channel Switches, 
the Cisco MDS 9506, 9509, 9216/9216A, 9216i, 9120, and the 9140. 



Note: Refer to the following Web page for the latest information on supported C-Series Fibre 
Channel switch models and firmware versions: 

http://hl 8006. wwwl .hp.com/storage/saninfrastructure.html 



Table 21 : HP C-Series Product Line Switches 





MM 


ware Version 


Maximum Number of 
Fibre Channel Ports 


MDS 9506 






128 


MDS 9509 


1.2.1a, 1.2.1b, 
1.3.4a, 2.0.1b 

HP recommends FW 
version 1 ,3.4a or 


224 


MDS 9216/9216A 


48 


MDS 9216i 


46 


MDS 9120 




above 


20 


MDS 9140 






40 



SAN Fabric Rules - C-Series Fabric Product Line 

The following rules are for SAN fabrics implemented with the switch FW versions listed in 
Table 2 1 , unless otherwise stated. 

1. The MDS 9506 is supported with up to 128 ports, over 4 slots (4-32 port modules). The 
32-port module for the MDS 9000 product line utilizes 3.2:1 oversubscription for 
optimized connectivity of low to mid-range host devices and low-end tape libraries. HP 
recommends the use of the 16 port module for performance intense host applications, disk 
storage systems, and high-end tape libraries, as well as ISLs. This will decrease the actual 
port count of the switch. 
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2. The MDS 9509 is supported with up to 224 ports, over 7 slots, each filled in with a 7-port 
to 32-port module. The 32-port module for the MDS 9000 product line utilizes 3.2:1 
oversubscription for optimized connectivity of low to mid-range host devices and low-end 
tape libraries. HP recommends the use of the 16 port module for performance intense host 
applications, disk storage systems, and high-end tape libraries, as well as ISLs. This will 
decrease the actual port count of the switch. 

3. The MDS 9216/9216A has a basic configuration with 16 ports. It has an expansion slot 
that supports either a 16 or a 32 port Fibre Channel card. 

4. The MDS 9216i has a basic configuration with 14 Fibre Channel ports and 2 iSCSI/FCIP 
ports. It has an expansion slot that supports any one of the following modules; 16 port 
Fibre channel, or 32 port Fibre Channel card. 

5. The MDS 9120 is a fixed 20 port configuration. It has 4 full rate ports and 16 
oversubscribed ports. 

6. The MDS 9140 is a fixed 40 port configuration. It has 8 full rate ports and 32 
oversubscribed ports. 

7. The MDS 9506, MDS 9509, and MDS 9216 models also support the IP Storage services 
8-Port lxGE module and the IP Storage services 4-Port lxGE module that both provide 
integrated FCIP and iSCSI functionality. HP recommends the use of these blades for 
integrated deployment and FCIP implementation for business continuity applications. 

8. The MDS 916/9216A, MDS 9216i, MDS 9506, and MDS 9509 also support the 
Multiprotocol Service Module (14 Fibre Channel ports & 2 iSCSI/FCIP ports). 

Fabric and Switch Model Maximums - C-Series Product Line 

The fabric maximums listed are for C-Series switch model SAN fabrics utilizing HP 
XP128/1024, XP48/512, VA71 10/7410, Enterprise Virtual Array (EVA), EMA/ESA12000, 
EMA16000, MA/RA8000, MA6000, MSA1000, and MSA1500 storage systems. Refer to 
Chapter 4 for specific operating system support for each storage system type based on the 
switch product line used. 

In general, these fabric rules also apply to Continuous Access XP, EVA, and DRM for 
EMA/ESA12000, EMA16000, and MA/RA8000 storage systems. Refer to Chapter 4 of this 
guide and the Continuous Access EVA Design Reference Guide for additional details. 

1. Up to 24 MDS switches with up to 3000 ports total in a SAN fabric 

2. Up to 12 MDS 9506 or MDS 9509 Directors with up to 10 MDS 9216/9216A/9216L 9120, 
or 9140 Fabric switches. 

3. Up to 12 MDS 9506 or 9509 Directors in an all Director fabric 
Table 22: C-Series Fabric and VSAN/lnter-VSAN Routing Maximums 



Maximum number of total ports per fabric 


3000 


Maximum number of user ports (device ports) per fabric 


2500 


Maximum number of hops per fabric 


7 


Maximum number of VSANs per fabric 


40 


Maximum number of inter-VSAN shared devices per routed fabric 


2000 


Maximum number of inter-VSAN zones per routed fabric 


200 
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Zoning and VSANs 

Table 23 indicates how zoning is implemented for the Cisco MDS switches. 

1. In HP C-Series SANs, Virtual Storage Area Networks (VSANs) are defined as separate 
instances of all fabric services, including address space. HP supports 40 VSANs per 
physical fabric. 

2. The maximum number of zones across all VSANs is 2048. 

3. The maximum number of zone members across all VSANs is 20,000. 

4. The maximum number of ports in VSAN when using inter- VSAN routing is 3000. 

5. The maximum number of devices that can be shared in a VSAN using inter- VSAN routing 
is 2000 



Table 23: Zone Types on HP fabric switches 



Switch Models 


Configuration 


Enforcement 


Comments 




Define zones using all 
domain#/port# 






Cisco MDS 9506 








Cisco MDS 9509 

Cisco MDS 92 1 6/92 1 6A/92 1 6i 

Cisco MDS 9120 


Define zones using all 
WWNs 


Access 

authorization at 
frame level in 
hardware 


HARD 
zoning 


Cisco MDS 9140 


Define zones using a 
combination of 
domain#/ port# and 
WWNs 







Mixed Storage Common SAN Rules 

Refer to Chapter 4 for specific storage rules related to the C-Series switches. 

C-Series High Availability VSAN Management Configuration 



Management 
Station 



Server 






Figure 37: C-Series High Availability VSAN Management Configuration 
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Figure 37 shows a typical high availability configuration with server and storage connections 
to different fabrics for high availability. This provides two paths for data access between 
servers and storage. The addition of an ISL between Fabric A and Fabric B allows both fabrics 
to be managed through a single management station. This configuration is classified as a level 
3 high availability configuration. Refer to Chapter 2, "Data Availability in a SAN", page 63 for 
information about levels of data availability in a SAN. 

Supported Switch Models - M-Series Fabric Product Line 

HP supports a range of 1 Gbps and 2 Gbps M-Series Fabric product line Fibre Channel switch 
models. These switch models represent products supported by both pre-merger HP and 
pre-merger Compaq. The relationship between the pre-merger and post-merger switch 
products is shown in Table 24. Refer to the section, SAN Fabric Rules - M-Series Fabric 
Product Line, for specific switch model support rules. 



Note: Refer to the following Web page for the latest information on supported M-Series Fibre 
Channel switch models and firmware versions: 

http://hl 8006. wwwl .hp.com/storage/saninfrastructure.html 



Table 24: HP StorageWorks M-Series Product Line Switches 



HP StorageWorks Switch Name 




umber of 
Ports 



HP StorageWorks edge switch 2/12 


06.02.00-22 


4 to 12 


HP StorageWorks edge switch 2/16 


16 


HP StorageWorks edge switch 2/24 


8 to 24 


HP StorageWorks edge switch 2/32 


16 to 32 


HP StorageWorks director 2/64 


32 to 64 


HP StorageWorks director 2/140 


64 to 140 


HP Switch Name 


Compaq Switch Name 




Number of 
Ports 


N/A 


McDATA ES-30 16 
(Compaq reseller) 


06.02.00-22 


16 


N/A 


McDATA ES-3032 
(Compaq reseller) 


32 


McDATA ED-5000 (McDATA reseller) 


04.01.01-2 


32 


HP Director FC-64 


Compaq StorageWorks 
SAN Director 64 


06.02.00-22 


64 



SAN Fabric Rules - M-Series Fabric Product Line 

All switch models shown in Table 24 are supported in the HP StorageWorks SAN provided 
that the same firmware versions and switch settings are utilized for all models with the 
exception of the ED-5000. The ED-5000 utilizes a unique firmware version, this version is 
compatible with the other switch model versions. The default switch settings for this family of 
switches from pre-merger HP and pre-merger Compaq are the same. 
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Fabric and Switch Model Maximums - M-Series Fabric Product Line 

The fabric maximums listed are for M-Series switch model SAN fabrics utilizing HP 
XP12000, XP128/1024, XP48/512, XP256, VA7100, VA7110, VA7400, VA7410, Enterprise 
Virtual Array (EVA), EMA/ESA12000, EMA16000, MA/RA8000, MA6000, or 
MSA1500/1000 storage systems. Refer to Chapter 4 for specific operating system support for 
each storage system type based on the switch product line used. 

In general, these fabric rules also apply to Continuous Access XP, EVA, and DRM for 
EMA/ESA12000, EMA 16000, and MA/RA8000. Refer to Chapter 4 of this guide and the 
Continuous Access EVA Design Reference Guide for additional details. 

The following rules are for SAN fabrics implemented with the switch FW version listed in 
Table 24 and 07.x or 08.x HAFM. The ED-5000 requires 04.01.00-16 switch FW and a 
minimum of 04.02.00 HAFM. 

1. Up to 24 switches with up to 1632 total ports and a maximum of 1024 user ports are 
supported in a single SAN fabric. Each fabric may contain any combination of supported 1 
Gbps and 2 Gbps switch models listed, provided the individual switch model fabric limits 
listed below are not exceeded. 

— HP StorageWorks director 2/64 and HP StorageWorks director 2/140 - maximum of 8 
Directors total per fabric 



Note: With eight fully populated HP StorageWorks director 2/140's, it is physically possible to 
exceed the 1024 user port maximum. The restriction is in the zoning configuration, as only 1024 
unique zone members (user ports) can be configured. However, the remaining ports can be used as 
Inter Switch Link (ISL) connections in the fabric. 

— Up to 16 switches with up to 512 total ports in a single fabric that includes one or 
more McDATA ED-5000 Directors. 

2. Up to 3 switch hops (4 switches) maximum between any two devices in a SAN fabric. 

3. Within a single fabric where switches are interconnected, each switch must have a unique 
domain number (Domain ID) and a unique World Wide Name (WWN). All switch 
configuration parameters in each switch must be the same. 



Note: Do not configure any switches with a domain ID of 8. HP systems reserve domain 8 for Private 
Loop devices. 



4. Zoning maximums - Table 25 lists the configuration limits for zones members, zones, and 
zone sets in the HAFM. 



Table 25: Zoning configuration supported limits for M-Series 





Firmware 05.xx.xx 


Firmware 06.xx.xx 


Zoning Parameter 


(Maximum) 


(Maximum) 


Number of zone members in a zone 


1024 


2048 


Number of zones in a zone set 


1024 


1024 


Number of unique zone members in a zone set 


1024 


2048 


Total number of zone members in a zone set (where 
a zone member can be in multiple zones) 


2048 


4096 


Characters per zoning name 


32 


32 
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Table 25: Zoning configuration supported limits for M-Series 



Zoning Parameter 


Firmware 05.xx.xx 
(Maximum) 


Firmware 06.xx.xx 
(Maximum) 


Number of unique zone members in HAFM Zoning 
Library 


2048 


2048 


Number of zones in HAFM Zoning Library 


1024 


1024 


Number of zone sets in HAFM Zoning Library 


64 


64 


Number of end ports 


1024 


1024 


Number of devices supported (including loop 
devices) 


1024 


1024 



5. Any mix of servers and storage systems is allowed in a SAN provided the specific 
platform, operating system, and storage system fabric limits and rules are followed. Refer 
to the appropriate sections in this guide and the documentation listed in the section 
"Related Documents" in the Preface. 

6. HP requires that all switches in a single fabric or multi-fabric SAN use the same switch 
firmware revision for the models that utilize the same firmware versions. Two successive 
fabric firmware versions can be temporarily used in one fabric or multiple fabrics in a 
SAN during switch firmware rolling upgrades. 

7. For M-Series Fabric product line, Table 24 Fibre Channel switches - Up to 7 switches 
configured in a ring with a Ring SAN fabric topology. 

General ISL Rules - All Fabric Product Lines 

When designing a fabric using 8-port, 16-port, 24-port, and 32-port switches, all ports may be 
used as ISLs with a maximum of one half of the total port count as ISLs to the same 
destination (may be license limited on some switch models). This feature is used in the 
Surestore FC Switch 6164 and StorageWorks SAN Switch Integrated 32 and 64 
configurations, which are made up of multiple 16-port switches in an integrated mechanical 
chassis package. However, there are restrictions in the use of ISLs on some higher port count 
Fibre Channel switch models, as shown in Table 26. 



Table 26: Number of ISLs for Fibre Channel switch products 



HP Switch 

Product Name Compaq Switch Name 


Total Number of 
Available Users Ports 


Number of Ports That May Be Used as 
ISLs 


HP StorageWorks SAN Director 2/128 


128 


128 


HP StorageWorks Core Switch 2/64 


64 per switch, 
1 28 per chassis 


64 per switch, 1 28 per chassis 


MDS 9000 32 Port Fibre Channel Module 


32 


8 Note: 3 User ports are disabled for each 
port configured as ISL. HP recommends the 
MDS 9000 16 Port Fibre Channel Module 
for ISL connections 


MDS9120 


20 


4 Note: for every ISL port configured 
beyond the initial 4, 3 User ports must be 
disabled. 


MDS 9140 


40 


8 Note: for every ISL port configured 
beyond the initial 8, 3 User ports must be 
disabled. 


McDATA ED-5000 


32 


4 
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Table 26: Number of ISLs for Fibre Channel switch products 



HP Switch 
Product Name 


Compaq Switch Name 


Total Number of 
Available Users Ports 


Number of Ports That May Be Used as 
ISLs 


Director FC64 


StorageWorks SAN Director 64 


64 


32 


HP StorageWorks Director 2/64 


64 


48 or 75% of installed ports 


HP StorageWorks Director 2/140 


140 


70 or 50% of installed ports 



Heterogeneous/Interoperable SAN Fabrics 

HP supports three configurations of interoperable heterogeneous SAN fabrics: 

■ SAN fabric with a mix of B-Series and M-Series switch models. 

■ SAN fabric with a mix of B-Series and C-Series switch models. 

■ SAN fabric with a mix of C-Series and M-Series switch models. 

HP also supports a dual heterogeneous SAN where one fabric consists exclusively of B-Series 
Fibre Channel switches, and a second fabric consists exclusively of M-Series Fibre Channel 
switches. 

Dual Heterogeneous SAN Fabrics 

Dual Heterogeneous SAN Fabrics are supported where one SAN fabric consists exclusively of 
the Fibre Channel switches listed in Table 18, HP StorageWorks B-Series Product Line 
Switches and the second SAN fabric consisting exclusively of the switches listed in Table 24, 
HP StorageWorks M-Series Product Line Switches. 

While each fabric could be designed to the maximum configuration for the applicable HP 
StorageWorks Series of Fibre Channel switches, good practice is to use the same configuration 
on both SAN fabrics. Symmetry is not a requirement, however, as a general rule, this 
maintains balanced SAN performance and also aids in understanding the configuration. In 
order to accomplish this, both fabrics should be designed to the least common design 
maximums between the two series of Fibre Channel switches. For example; HP StorageWorks 
B-Series product line Switches support 7 switch hops and HP StorageWorks M-Series product 
line Switches support 3 switch hops. 

■ When implementing a SAN with these two switch product lines, both fabrics should be 
designed with a maximum of 3 switch hops. 

■ For HA applications, since servers will connect to both fabrics, common HBA/driver, 
multi-path software, and storage array firmware versions support are required. 



Note: Dual Heterogeneous SAN Fabrics are not supported on MSA1000/MSA1500. Continuous 
Access/DRM configurations are supported with Dual Heterogeneous SAN Fabrics, including 
configurations extended with FCIP. 



Interoperable SAN Fabrics 

B-Series with C-Series Switches 

Within a single fabric, C-Series Director and Fabric switch models intermixed with B-Series 
switch models. The specific rules and configuration settings for each type of SAN fabric are 
described in the Fabric Interoperability: Merging Fabrics Based on C-Series and B-Series 
Fibre Channel Switches Application Notes (AA-RV1ZA-TE). 
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B-Series with M-Series Switches 

Within a single fabric, HP supports M-Series switch models intermixed with B-Series SAN 
switch models. The specific rules and configuration settings for each type of SAN fabric are 
described in the Fabric Interoperability: Merging Fabrics Based on M-Series and B-Series 
Fibre Channel Switches Application Notes (AA-RUQQC-TE). 

C-Series with M-Series Switches 

Within a single fabric, HP supports C-Series switch models intermixed with M-Series switch 
models. The specific rules and configuration settings for each type of SAN fabric are described 
in the Fabric Interoperability: Merging Fabrics Based on C-Series and M-Series Fibre 
Channel Switches Application Notes (AA-RVH9B-TE). 

These documents are available at: 

http:/ /h 1 8006. wwwl .hp.com/ products/ storaqeworks/ san/ documentation.html 



Note: Refer to the Application Notes above for a list of specific products supported in interoperable 
SAN Fabrics. Interoperable SAN Fabrics are not supported on MSA1000 or in Continuous 
Access/DRM configurations. 



Third party switch support 

Third party switches and connectivity reliability have been verified through testing. However, 
support is subject to the following: 

■ No support for third party switch functionality. Customers will need to work directly with 
the third party switch support organization to fix a switch defect. 

■ HP will make a best effort attempt to help the customer resolve issues as they pertain to the 
HP supported products within the environment. 

■ HP can support switches and additional functionality resold through other vendors 
providing the customer purchases 3rd party support through the HP SAN Environmental 
Services Group (SAN-ES). Refer to http://3so.qr.hp.com/N3SQPSM/snes/PSPs.htm for 
additional information. 

Examples of third party switches are: 

■ EMC Connectrix ED- 1032 

■ InRange FC 9000 

1 and 2 Gbps Fabric Topology Recommendations 

There are no specific topology rules related to mixing of 1 and 2 Gbps components in a fabric. 
HP does however strongly recommend these guidelines be followed. 

■ When using both 1 and 2 Gbps switches in the same fabric, utilize 2 Gbps switches in the 
core for Core to SAN switch or Director to edge switch topologies. 

■ Connect 2 Gbps switches together to take advantage of the optional ISL Trunking feature 
when using Fibre Channel switch models that support this feature. 

■ Utilize 2 Gbps switches for connections to 2 Gbps capable devices. In general, for SANs 
with both 1 Gbps and 2 Gbps components, the transfer rate between devices and ports on 
switches is determined by the speeds supported by the individual ports that are connected. 
If two 2 Gbps devices or switch ports are connected together, the speed will be 2 Gbps for 
that segment in the fabric. If two 1 Gbps devices or switch ports are connected together the 
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speed will be 1 Gbps for that segment in the fabric. If a 2 Gbps and a 1 Gbps port are 
connected together the speed will be 1 Gbps for that segment. Refer to the section General 
Fabric Performance Recommendations in this chapter for more information. 

■ Zoning rules, SAN security, and SAN management for 2 Gbps switches are the same as 
for 1 Gbps switches. 

SAN Fabric Zoning Rules 

The fabric zoning feature is supported with all HP Fibre Channel switch models. Zoning can 
be used to logically separate devices and different hardware platforms and operating systems 
in the same physical SAN. Use of zoning is required under these specific conditions: 

■ When mixing different storage system models and servers in the same SAN fabric. Refer 
to Chapter 4, "Heterogeneous SAN Platform and Storage System Rules" for more 
information. 

■ When mixing different hardware platforms, operating systems or storage systems that are 
currently only supported in homogenous SANs, and it is unknown whether there are 
interaction problems. Refer to Table 32 for specific information about zoning in 
heterogeneous SANs. 

■ When there are known interaction problems between different hardware platforms or 
operating systems and specific storage system types. 

■ When the number of nodes or ports in the SAN fabric exceeds a storage system connection 
support limit. There is a connection limit for storage systems using the 
EVA5000/EVA3000 (HSV controller) or EMA/ESA/MA/RA (HSG60/80 controller). The 
version of VCS or ACS controller code determines the specific limit. 

■ Overlapped zones are supported. Refer to the specific platform and storage system rules 
for more information. 

■ The maximum number of zones supported in a SAN fabric is based on the specific switch 
product line in use. Refer to the fabric and switch model maximums for each switch 
product line for more information. 

■ When using a SAN Management Appliance (see the next section.) 

Storage Management Appliance Rules and Recommendations 

Whenever a Storage Management Appliance is placed in a fabric it is recommended that a 
dedicated storage management zone be created. This zone is specifically for the Storage 
Management Appliance and the elements it is to monitor and manage. 

Currently, the Storage Management Appliance communicates with the EVA5000/EVA3000 
(HSV controller) and EMA/ESA12000, EMA 16000, MA/RA8000, MA6000 storage systems 
(HSG controller) in-band, that is, within the Fibre Channel fabric itself. It is not necessary or 
recommended to include either the switch WWNs or server HBA WWNs in this zone. 
Management communication to these devices from the Storage Management Appliance is 
done out-of-band or outside the fabric via TCP/IP. 

For example, create a zone called SANAPP_l_ZONE that would contain the Appliance host 
bus adapter WWN and the WWNs of all the HSG or HSV controllers managed by this Storage 
Management Appliance. Because fabric devices can be in multiple zones, this will have no 
effect on other zones containing the same HSG and HSV controller WWNs. 

1. Within the same fabric, EVA5000/EVA3000s with V2.0x and V3.0x can be managed by 
the same instance of Command View EVA. A separate appliance is needed to manage an 
EVA5000/EVA3000 with V 1 .Ox. 
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2. Any EVA storage system can only have one active SMA managing it. Any standby SMA 
can be powered on, but the Command View EVA or Continuous Access user interface 
must not control the storage system. For further information, refer to the 

HP StoraqeWorks Continuous Access EVA Operations Guide available on the HP web. 

3. A Storage Management Appliance is required to manage EVA5000/EVA3000s. For 
EMA/ESA12000, EMA16000, MA/RA8000, MA6000 storage systems, it is 
recommended that a Storage Management Appliance be used to manage the SAN when a 
fabric contains more than four Fibre Channel switches. It is required when using the HSG 
element manager. 

4. Multiple appliances per fabric are allowed as long as only one appliance is accessing a 
controller pair at a time. Zoning is required to actively isolate an appliance and its 
controllers from other appliances and their controllers. 

5. Element Manager for HSG may manage up to 25 HSG controller pairs per appliance. 
Command View for HSV may manage up to 16 HSV controller pairs per appliance. 

6. The HSV and HSG Element Managers can operate in a dual fabric configuration. 

7. A license key is required for each EVA5000/EVA3000 managed by the Storage 
Management Appliance. Additional licenses are required to take advantage of value added 
software functionality. 

8. For SANs with more than 1024 HBAs, an HSV controller must be zoned so that it can see 
no more than 1024 HBAs. It may be necessary to add a zone to a SAN to satisfy the 1024 
HBA limit. 

Refer to Chapter 4, "Heterogeneous SAN Platform and Storage System Rules", for rules about 
mixing specific platforms in a Heterogeneous SAN without the need for fabric zoning. 
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SAN Component Interconnect Descriptions and Rules 

The following sections describe rules for SAN component interconnects-switch port 
interfaces and physical cabling. 

Fibre Channel Switch Interface Usage Descriptions 

■ E-Port interface for Fibre Channel switch to switch, switch to gateway, or switch to bridge 
connectivity (also referred to as ISLs). 

■ EX-Port - Enhanced E-Port used to connect a Fibre Channel router to an edge fabric 

■ F-Port interface for fabric attached device-initiators (host bus adapters) and targets 
(storage ports) 

■ FL-Port interface for public loop fabric-aware, 24-bit Fibre Channel addressable 
devices-initiators or Compaq FC-AL Switch 8 SAN attachment 

■ FL-Port interface for private loop 8-bit Fibre Channel addressable devices-private FC-AL 
initiators and targets. This requires use of the B-Series product line switches QuickLoop 
Feature. This is a license-enabled feature on certain switch models, refer to the specific 
switch model product documentation for information about support of this feature. 
Typically this feature is only required when a specific platform can only be configured 
with a private FC-AL host bus adapter driver. 

■ G-Port, default interface when no devices appear to be attached to the port. 

Access with QuickLoop 

The QuickLoop switch feature allows private FC-AL initiators and targets configured in a 
QuickLoop to communicate with each other through the switch. Since all initiators configured 
inside a QuickLoop are private they cannot communicate with targets outside of the 
QuickLoop. QuickLoop is only supported on the B-Series product line of Fibre Channel 
switch models when used with MA6000, MA8000, RA8000, EMA 12000/1 6000, and 
ESA12000 RAID Array systems. 

Fiber Optic Interconnect Rules 

1. 2 Gbps components utilize industry standard "LC" connectors for the Fibre Channel 
optical connections. 1 Gbps components utilize industry standard "SC" connectors. Cables 
and adapters are available with SC connectors on one end and LC connectors on the other 
end. Refer to the 2 Gbps switch QuickSpecs documents for additional information. 



Note: Certain fiber optic cable configurations may require the use of SC or LC connector sleeves to 
couple two cable connector ends together such as when using wall jacks or connecting to existing 
pre-installed cables. Duplex couplers are available from various manufacturers. Use of these couplers 
is supported provided the overall cable losses specified in this chapter are not exceeded in the cable 
segment that includes the couplers. 



2. The minimum allowable bend radius of fiber optic cable is 25 mm for 50, 62.5, and 9 
micron fiber optic cable. 
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3. There is a minimum fiber optic cable segment length between Fibre Channel devices (a 
transmitter and a receiver). The minimum length is 0.5 meters for 50 and 62.5 micron 
cable and 2.0 meters for 9 micron cable. The minimum length does not apply to patch 
cords through a passive patch panel; it only applies to the total distance between the 
transmitter and receiver of the devices being connected through the patch panel. 

4. For longer 50-micron short wave multi-mode optical fiber cables up to 300 meters (2 
Gbps) or up to 500 meters (1 Gbps), a third party vendor must be contacted. The cables 
must be duplex, tight buffered multi-mode 50/125 um (Belcore GR-409 compliant) and 
the connectors must be SC or LC duplex low metal Belcore and IEC compliant. 

5. For 9-micron long wave single-mode optical fiber cables up to 100 km (1 Gbps) or 35 km 
(2 Gbps), a third party vendor must be contacted. The cables must be duplex, tight 
buffered, single-mode 9/125 um (Belcore GR-409 compliant) and the connectors must be 
SC or LC duplex low metal (NTT-SC Belcore 326, IEC-874-19 SC compliant). 

6. The mixing of 9-micron, 50-micron, and 62.5-micron fiber cables in the same cable 
segment is not supported. 

7. Extended Fabrics - See Chapter 7, SAN Extension. Also refer to the Extended Fabric 
User Guide, AA-RR7QA-TE, for more information. 

4 Gbps Fiber Optic Interconnects/Distance Rules 

1. Up to 150 meters maximum distance per cable segment between switches using 50/125 
micron multi-mode fiber optic cable and short wavelength SFPs. 

2. Up to 70 meters maximum distance per cable segment between switches using 62.5/125 
micron multi-mode fiber optic cable and short wavelength SFPs. 



Note: Information on the use of 62.5 micron fiber optic cable is provided to facilitate use of 
previously installed cable. HP recommends 50 micron fiber optic cable for any new installation 
requiring multi-mode fiber. 



2 Gbps Fiber Optic Interconnects/Distance Rules 

1. Up to 300 meters maximum distance per cable segment between devices and switches or 
switches and switches using 50/125 micron multi-mode fiber optic cable and short 
wavelength SFPs 

2. Up to 150 meters maximum distance per cable segment between devices and switches or 
switches and switches using 62.5/125 micron multi-mode fiber optic cable and short 
wavelength SFPs 



Note: Information on the use of 62.5 micron fiber optic cable is provided to facilitate use of 
previously installed cable. HP recommends 50 micron fiber optic cable for any new installation 
requiring multi-mode fiber. 



3. Up to 10 km (6.2 miles) maximum distance between two switches or B-Series switches 
and the MP Router using 9/125 micron single-mode fiber optic cable and long wavelength 
SFPs. 

4. Up to 35 km (21.7 miles) maximum distance between two switches using 9/125 micron 
single-mode fiber optic cable and extended reach SFPs. 
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5. For B-Series product line switches: 

■ Maximum of 160 km (100 miles) total distance across the SAN using multiple segments. 
This can be implemented using four 35km segments and two 10 km segments (6 hops.) In 
all cases the individual segments must be configured with the proper SFP and cable type 
and a maximum of 7 hops must not be exceeded across the SAN. 

■ ISL connections up to 10 km are supported between 1Gb and 2Gb switches at the "L0" 
portcfglongdistance setting only. 

6. For C-Series product line switches: 

■ Maximum of 160 km (100 miles) total distance across the SAN using multiple segments. 
In all cases the individual segments must be configured with the proper SFP and cable type 
and a maximum of 3 hops. 

7. For M-Series product line switches: 

■ Maximum of 105 km (64 miles) total distance across the SAN using multiple segments. 
This can be implemented using three 35 km segments (3 hops.) In all cases the individual 
segments must be configured with the proper SFP and cable type and a maximum of 3 
hops must not be exceeded across the SAN. 



Note: WDM may be used to increase the switch-to-switch distance beyond the distances listed 
above. Refer to Chapter 7, SAN Extension. 



Note: Refer to the Data Replication Manager (DRM) solution documentation, including the DRM 
Design Guide, for specific interconnect and distance rules related to DRM configurations. See: 

http://hl 8000. wwwl .hp.com/ products/sanworks/ drm/ documentation.html 



Note: Refer to the Continuous Access EVA solution documentation, including the Continuous Access 
EVA Design Reference Guide, for specific interconnect and distance rules related to Continuous 
Access EVA configurations. See: 

http://hl 8006. wwwl .hp.com/ products/storage/software/ conaccesseva/index.html 



1 Gbps Fiber Optic Interconnects/Distance Rules 

1. Up to 500 meters maximum distance per cable segment between devices and switches or 
switches and switches using 50/125 micron multi-mode fiber optic cable and short 
wavelength GBICs or GLMs 1 . 

2. Up to 200 meters maximum distance per cable segment between devices and switches or 
switches and switches using 62.5/125 micron, multi-mode fiber optic cable and short 
wavelength GBICs or GLMs 1 . 



Note: Information on the use of 62.5 micron fiber optic cable is provided to facilitate use of 
previously installed cable. HP recommends 50 micron fiber optic cable for any new installation 
requiring multi-mode fiber. 



1 . GLMs are used in the HSG60 (MA6000) and HSG80 (MA/RA8000, EMA/ESA1 2000, EMA1 6000) 
storage controllers 
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3. For B-Series product line switches: 

■ Up to 10 km (6.2 miles) maximum distance between any two switches using 9/125 micron 
single-mode fiber optic cable and long wavelength GBICs. 

■ Up to 100 km (62 miles) distance between any two switches using 9/125 micron 
single-mode fiber optic cable and very long distance GBICs. A maximum of one 100 km 
very long distance segment per SAN. 

■ Maximum of 160 km (100 miles) total distance across the SAN using multiple segments. 
This can be implemented using a single 100 km segment and six 10 km segments (7 hops) 
or other combinations such as two 50 km segments and five 10 km segments (7 hops). In 
all cases the individual segments must be configured with the proper GBICs and cable 
type and the maximum of 7 hops must not be exceeded across the SAN. 

■ ISL connections up to 10 km are supported between 1Gb and 2Gb switches at the "L0" 
portcfglongdistance setting only. 

4. For C-Series product line switches: 

■ Maximum of 30 km (18.6 miles) total distance across the SAN using multiple segments. 
This can be implemented using three 10 km segments (3 hops.) In all cases, each segment 
must be configured with the proper SFP and cable type and a maximum of 3 hops. 

5. For M-Series Fabric product line switches: 

■ Up to 10 km (6.2 miles) maximum distance between any two switches using 9/125 micron 
single-mode fiber optic cable and long wavelength SFPs. 

■ Up to 35 km (21.7 miles) maximum distance between two switches using 9/125 micron 
single-mode fiber optic cable and extended reach SFPs. 

■ Maximum of 105 km total distance across the SAN using multiple segments. This can be 
implemented using three 35 km segments (3 hops.) In all cases the individual segments 
must be configured with the proper SFP and cable type and a maximum of 3 hops must not 
be exceeded across the SAN. 



Note: WDM may be used to increase the switch-to-switch distance beyond the distances listed 
above. Refer to Chapter 7, SAN Extension. 



Note: Refer to the Data Replication Manager (DRM) solution documentation, including the DRM 
Design Guide, for specific interconnect and distance rules related to DRM configurations. See: 

http:/ /hi 8000. wwwl .hp.com/ products/ sanworks/ drm/ documentation.html 



Note: Refer to the Continuous Access EVA solution documentation, including the Continuous Access 
EVA Design Reference Guide, for specific interconnect and distance rules related to Continuous 
Access EVA configurations. See: 

http://hl 8006. wwwl .hp.com/ products/storaqe/software/ conaccesseva/index.html 
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Fiber Optic Cable Loss Budgets 

The information in this section is based on the Fibre Channel Physical Interface Specification. 
Refer to the specification document for more information. The maximum distances specified 
are based on the use of nominal bandwidth fiber optic cable. This specifies modal bandwidth 
of 500 MHz-km for 50 micron fiber optic cable, and 200 MHz-km for 62.5 micron fiber optic 
cable. 



Note: Media losses are not specified due to variances between different fiber optical cable 
manufacturers. In all cases the specification that must be followed is the total channel insertion loss, 
which includes media losses. 



Note: Information on the use of 62.5 micron fiber optic cable is provided to facilitate use of 
previously installed cable. HP recommends 50 micron fiber optic cable for any new installation 
requiring multi-mode fiber. 



Table 27: Optical Cable Losses 



Speed 


Cable 


Maximum 
Distance 


Total Channel 
Insertion Loss 1 


Loss per Mated 
Connector Pair 


Notes 


4 Gbps 


62.5/125 
micron 


70 meters 


1 .78 db 


0.75 db 




4 Gbps 


50/125 
micron 


1 50 meters 


2.06 db 


0.75 db 




2 Gbps 


62.5/125 
micron 


1 50 meters 


2.1 dB 


0.75 dB 




2 Gbps 


50/125 
micron 


300 meters 


2.62 dB 


0.75 dB 




2 Gbps 


9/125 
micron 


10 km 


7.8 dB 


0.75 dB 




2 Gbps 


9/125 
micron 


35 km 


19 dB 


0.75 dB 




1 Gbps 


62.5/125 
micron 


200 meters 


3.0 dB 


0.75 dB 




1 Gbps 


50/125 
micron 


500 meters 


3.85 dB 


0.75 dB 




1 Gbps 


9/125 
micron 


10 km 


7.8 dB 


0.75 dB 




1 Gbps 


9/125 
micron 


100 km 


21.5 dB 


0.75 dB 


A minimum loss of 
8 dB is required 



1. Channel insertion loss is the combined passive loss from connectors, splices, and media between the 
transmitter and receiver. 



Use of optical fiber patch panels is supported provided the total channel insertion loss between 
the transmitter and receiver for the cable segment routed through the patch panel does not 
exceed the maximum listed for the connector and cable type in use. 
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Table 28: Storage Product Interconnect/Transport Support 







Storage Product 




Interface/ 
Transport 


Heterogeneous SAN 


DRM and Continuous 
Access EVA 


Enterprise Backup 
Solutions (EBS) 


2 Gbps Fibre 
Channel via 50 
micron multi-mode 
fiber optic cable and 
short-wave SFPs 


Up to 300 meters per cable 
segment. Refer to 2 Gbps Fiber 
Optic Interconnects/Distance Rules, 
page 103, for maximum supported 
distance across the SAN. 


Up to 300 meters per cable 
segment. Refer to 2 Gbps Fiber 
Optic Interconnects/Distance 
Rules, page 103, for maximum 
supported distance across the 
SAN. 


Up to 300 meters per cable 
segment. 

Refer to 2 Gbps Fiber Optic 
Interconnects/ Distance 
Rules, page 1 03, for 
maximum supported 
distance across the SAN. 


2 Gbps Fibre 
Lnannel via o/.o 
micron multi-mode 
fiber optic cable and 
short-wave SFPs 


Up to 1 50 meters per cable 
segment. Refer to 2 Gbps Fiber 
Optic Interconnects/Distance Rules, 
page 103, for maximum supported 
distance across the SAN. 


Up to 150 meters per cable 
segment. Refer to 2 Gbps Fiber 
Optic Interconnects/Distance 
Rules, page 103, for maximum 
supported distance across the 
SAN. 


Up to 1 50 meters per cable 
segment. 

Refer to 2 Gbps Fiber Optic 
Interconnects/Distance 
Rules, page 1 03, for 
maximum supported 
distance across the SAN. 


2 Gbps Fibre 
Channel via 9 micron 
single-mode fiber 
optic cable and 
long-wave SFPs 


Up to 10 km per cable segment. 
Refer to 2 Gbps Fiber Optic 
Interconnects/Distance Rules, 
page 103, for maximum supported 
distance across the SAN. 


Up to 10 km per cable 
segment. Refer to 2 Gbps Fiber 
Optic Interconnects/Distance 
Rules, page 103, for maximum 
supported distance across the 
SAN. 


Up to 10 km per cable 
segment. 

Refer to 2 Gbps Fiber Optic 
Interconnects/Distance 
Rules, page 1 03, for 
maximum supported 
distance across the SAN. 


2 Gbps Fibre 
Channel via 9 micron 
single-mode fiber 
optic cable and 
extended reach SFPs 


Up to 35 km per cable segment. 
Refer to 2 Gbps Fiber Optic 
Interconnects/Distance Rules, 
page 103, for maximum supported 
distance across the SAN. 


Up to 35 km per cable 
segment. Refer to 2 Gbps Fiber 
Optic Interconnects/Distance 
Rules, page 103, for maximum 
supported distance across the 
SAN. 


Up to 35 km per cable 
segment. Refer to 2 Gbps 
Fiber Optic 

Interconnects/Distance 
Rules, page 1 03, for 
maximum supported 
distance across the SAN. 


1 Gbps Fibre 
Channel via 50 
micron multi-mode 
fiber optic cable and 
short-wave GBICs 


Up to 500 meters per cable 
segment. Refer to 1 Gbps Fiber 
Optic Interconnects/Distance Rules, 
page 104, for maximum supported 
distance across the SAN. 


Up to 500 meters per cable 
segment. Refer to 1 Gbps Fiber 
Optic Interconnects/Distance 
Rules, page 104, for maximum 
supported distance across the 
SAN. 


Up to 500 meters per cable 
segment. 

Refer to 1 Gbps Fiber Optic 
Interconnects/Distance 
Rules, page 1 04, for 
maximum supported 
distance across the SAN. 


1 Gbps Fibre 
^.nannei via oz.j 
micron multi-mode 
fiber optic cable and 
short-wave GBICs 


Up to 200 meters per cable 
segmenT. ixerer to i oops nuer 
Optic Interconnects/Distance Rules, 
page 104, for maximum supported 
distance across the SAN. 


Up to 200 meters per cable 
segmenT. ixerer to i oops nuer 
Optic Interconnects/Distance 
Rules, page 104, for maximum 
supported distance across the 
SAN. 


Up to 200 meters per cable 
segmenr. 

Refer to 1 Gbps Fiber Optic 
Interconnects/Distance 
Rules, page 1 04, for 
maximum supported 
distance across the SAN. 


1 Gbps Fibre 
Channel via 9 micron 
single-mode fiber 
optic cable and 
long-wave GBICs 


Up to 10 km per cable segment. 
Refer to 1 Gbps Fiber Optic 
Interconnects/ Distance Rules, 
page 104, for maximum supported 
distance across the SAN. 


Up to 10 km per cable 
segment. Refer to 1 Gbps Fiber 
Optic Interconnects/Distance 
Rules, page 104, for maximum 
supported distance across the 
SAN. 


Up to 10 km per cable 
segment. 

Refer to 1 Gbps Fiber Optic 
Interconnects/ Distance 
Rules, page 1 04, for 
maximum supported 
distance across the SAN. 
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Table 28: Storage Product Interconnect/Transport Support (Continued) 







Storage Product 




Interface/ 
Transport 


Heterogeneous SAN 


DRM and Continuous 
Access EVA 


Enterprise Backup 
Solutions (EBS) 


1 Gbps Fibre 
Channel via 9 micron 
single-mode fiber 
optic cable and very 
long distance GBICs 


Up to 100 km per cable segment. 
Refer to 1 Gbps Fiber Optic 
Interconnects/Distance Rules, 
page 104, for maximum supported 
distance across the SAN. 


Up to 100 km per cable 
segment. Refer to 1 Gbps Fiber 
Optic Interconnects/Distance 
Rules, page 104, for maximum 
supported distance across the 
SAN. 


Up to 100 km per cable 
segment. 

Refer to 1 Gbps Fiber Optic 
Interconnects/ Distance 
Rules, page 1 04, for 
maximum supported 
distance across the SAN. 


1 Gbps and 2 Gbps 
Fibre Channel via 
Wavelength Division 
Multiplexing (WDM) 


Supported up to 160 km total 
distance across the SAN. Refer to 
the SAN Extension Chapter. 


Supported up to 160 km total 
distance across the SAN. Refer 
to the SAN Extension Chapter. 


Supported up to 160 km 
total distance across the 
SAN. Refer to the SAN 
Extension Chapter. 


ATM over single 
Tl/El Wide Area 
Network (WAN) 


Not Supported 


Supported, refer to page 1 67 
Supported on DRM using an FC 
to ATM convertor with no limit 
on delay, and as the intersite 
backbone for Continuous 
Access EVA with an FC to IP 
convertor with up to 1 00 ms of 
IP network delay. Assumes a 
single Tl/El per fabric in a 
dual fabric. Not supported as a 
single ISL in a single fabric 
implementation. 


Not Supported 


ATM over single 
Tl/El WAN 
(Inverse Multiplexing) 


Not Supported 


Supported, refer to page 1 67 
Supported on DRM using an FC 
to ATM convertor with no limit 
on delay, and as the intersite 
backbone for Continuous 
Access EVA with an FC to IP 
convertor with up to 1 00 ms of 
IP network delay. Assumes a 
single Tl/El per fabric in a 
dual fabric. Not supported as a 
single ISL in a single fabric 
implementation. 


Not Supported 


ATM over T3/E3 
WAN 


Not Supported 


Supported 
Refer to page 1 67 
Supported on DRM using an FC 
to ATM convertor with no limit 
on delay, and as the intersite 
backbone for Continuous 
Access EVA with an FC to IP 
convertor with up to 1 00 ms of 
IP network delay. 


Not Supported 


ATM over fractional 
and/or shared 
T3/E3 and OC3 
WAN 


Not Supported 


Supported 
Refer to page 1 67 
Supported on DRM using an FC 
to ATM convertor with no limit 
on delay, and as the intersite 
backbone for Continuous 
Access EVA with an FC to IP 
convertor with up to 1 00 ms of 
IP network delay. 


Not Supported 
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Table 28: Storage Product Interconnect/Transport Support (Continued) 







Storage Product 




Interface/ 
Transport 


Heterogeneous SAN 


DRM and Continuous 
Access EVA 


Enterprise Backup 
Solutions (EBS) 


10/100 Copper 
Ethernet FC over IP 
(FCIP) 


Supported on EVA, 
EMA/ESA 12000, EMA 16000, 
MA/RA8000 and MA6000 with 
up to 100 ms of IP network delay. 
MSA1 000/MSA1 500 supported 
up to 160 km total Fibre Channel 
distance. 


Supported on DRM and 
Continuous Access EVA with up 
to 100 ms of IP network delay. 


Not Supported 


1 Gbps Optical 
Ethernet FC over IP 
(FCIP) 


Supported on EVA, 
EMA/ESA 12000, EMA 16000, 
MA/RA8000 and MA6000 with 
up to 100 ms of IP network delay. 
MSA1 000/MSA1 500 supported 
up to 160 km total Fibre Channel 
distance. 


Supported on DRM and 
Continuous Access EVA with up 
to 100 ms of IP network delay. 


Not Supported 


Fibre Channel 
Routing using the HP 
MP Router 


Supported on EVA, XP, VA, 

EMA/ESA 12000, EMA 16000, 

MA/RA8000, MA6000 

MSA1 000/MSA1 500 up to 1 60 
km total Fibre Channel distance. 
Maximum of 10 km distance 
between an MP Router and an edge 
fabric. 


See below when combining 
Fibre Channel Routing with 
FCIP. 


Supported 


FCIP SAN Extension 
using the HP MP 
Kouter 


Supported on EVA, XP, VA, 

EMA/ESA 12000, EMA 16000, 
k a a /da onnri k a a Annn 

MSAl 000/MSA1 500 up to 1 60 
km total Fibre Channel distance. 
Maximum of 10 km distance 
between an MP Router and an edge 
fabric. 


Supported on Continuous 
Access EVA and Continuous 
Access Ar witn up to I UUms ot 
IP network delay. Maximum of 
1 0 km distance between an MP 
Router and an edge fabric. 


Not Supported 


iSCSI bridging using 
the HP SR2122-2 IP 
storage router or the 
C-Series MDS IP 
Services Module 


Supported on EVA, XP, VA, 
EMA/ESA 12000, EMA 16000, 
MA/RA8000, MA6000, and 
MSAl 000/MSA1 500 supported 
up to 160 km total Fibre Channel 
distance. 


Not Supported 


Not Supported 


FCIP SAN extension 
using the HP 
SR21 22-2 IP storage 
router 


Supported on EVA, XP, VA, 
EMA/ESA 12000, EMA 16000, 
MA/RA8000 and MA6000 with 
up to 100 ms of IP network delay 
and up to 160 km total Fibre 
Channel distance. 
MSAl 000/MSA1 500 supported 
up to 160 km total Fibre Channel 
distance. 


Supported on DRM, Continuous 
Access EVA, and Continuous 
Access XP with up to 1 00ms of 
IP network delay. 


Supported with EBS with up 
to 1 00ms of IP network 
delay. 


FCIP SAN extension 
using the C-Series 
MDS IP Services 
Module 


Supported on EVA, XP, 
EMA/ESA 12000, EMA 16000, 
MA/RA8000 with up to 1 00 ms of 
IP network delay. 


Supported on DRM, Continuous 
Access EVA, and Continuous 
Access XP with up to 1 00ms of 
IP network delay. 


Not Supported 
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Note: For information on all HP supported DRM and Continuous Access SAN extension products 
refer to the HP StorageWorks Continuous Access And Data Replication Manager SAN Extensions 
available at: 

http://hl 8000. wwwl .hp.com/ products/storageworks/san/ documentotion.htm 



General Fabric Performance Recommendations 

The performance of an application on a heterogeneous SAN is usually seen from the 
perspective of "storage performance". The intervening SAN and competing workloads are not 
usually considered. The fact is that the performance of the storage will be dependent on the 
interaction of all the components and applications in the SAN. Some of the possible 
component limiting factors include the host CPU(s), Fibre Channel HBA, SAN topology, SAN 
traffic, RAID controllers, or the specific configuration of disks used behind the controllers. 
This is a dynamic workload environment and at any given moment any part of the SAN can 
dominate the performance. This complexity can be simplified if the environment is divided 
into the categories of servers, SAN infrastructure, and data storage devices. This document is 
primarily interested in the SAN infrastructure. There are issues with storage and servers that 
are unique to SAN implementations and we will address some of those as well. 

SAN Infrastructure Performance 

For the purposes of discussion assume a multi-switch fabric, since single switches always offer 
the highest performance with minimum latency. A multi- switch fabric has two factors that 
decrease overall fabric-wide infrastructure performance: 

■ latency through multiple switches (hops) 

■ oversubscription or congestion of ISLs 

Performance testing and measurement by HP has shown switch latency to be less than 5% 
(at 1 Gbps) of the time lost due to congestion of a full frame from another path. This implies 
that the number of switches and hops between devices is not a major factor for performance. 
However, as devices send frames through more switches and hops, the chances are increased 
that other traffic in the SAN may be routed over the same ISL/path. This may decrease 
performance due to oversubscription of a particular ISL/path that is serving multiple devices. 

Oversubscription has been determined to be the largest contributing factor to reduced Fibre 
Channel performance. When devices must contend for the same ISL or path, the best result 
will be that each competing device will receive 1/n of the available bandwidth on the path 
(where n is the number of contending devices). 

While the topology and size of the SAN have been seen to affect performance, staying within 
the rules and recommendations outlined in this guide minimizes these factors. The topology 
designs have been defined to accommodate a particular data access or data locality type. 
Recommendations on the number of ISLs based on device-to-device access ratios serve to 
ensure that adequate bandwidth is available across the SAN, minimizing oversubscription. 

HP recommends following these guideline in configuring your SAN. 

■ Whenever possible, devices that exchange the highest amount of data should be connected 
to the same Fibre Channel switch. 

■ For high bandwidth, the number of application servers should be balanced with storage by 
using as much one-to-one access as possible 

■ When devices exchanging data are on different switches: 
— Minimize the number of hops between devices 
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— For high bandwidth (large transfer size) applications, configure a maximum of two 
active storage controller ports per ISL 

— For high throughput (small transfer size) applications, configure a maximum of 20 
active storage controller ports per ISL 

— For mixed applications, configure a maximum of 4 active storage controller ports/ ISL 

Performance Considerations for Mixed 1 Gbps and 2 Gbps SAN Fabrics 

For SAN fabrics consisting of a mixture of 1 and 2 Gbps switches and devices, the individual 
fabric segment connections negotiate the speed at which specific devices communicate. The 
presence of 1 Gbps devices in a fabric will not force other independent 2 Gbps devices to a 
lower speed. That is, switch ports or user ports in a fabric capable of 2 Gbps will always 
communicate at their highest supported speed, 2 Gbps, when connected to other switch or user 
ports capable of 2 Gbps. The sustained data rate between the devices will be 200 MB/second 
assuming the entire path between the devices through the fabric is capable of 2 Gbps. 

If you have 1 Gbps and 2 Gbps devices connected to the same 2 Gbps switch, or if an 
intermediate path or route between devices is 1 Gbps, the switch will buffer data coming from 
the 2 Gbps device to prevent overrunning the 1 Gbps port or device. Over time the sustained 
data rate will be 100 MB/second. 

It is recommended that devices capable of 2 Gbps always be connected to 2 Gbps switches and 
other 2 Gbps devices. Ensure that the entire path or route between 2 Gbps devices consists of 2 
Gbps capable ports. 

On the C-Series switches ensure that Fibre-Channel Congestion Control (FCC) is enabled on 
all the switches. The FCC is a feature of the C-Series switches which allows these switches to 
intelligently regulate traffic across ISLs and ensure each host-target pair of devices will have 
the required bandwidth for data transfer. The C-Series switches also have the ability to 
prioritize frames originating from a particular port hence applications can be prioritized using 
the Quality of Service (QOS) feature. 

Performance Specifications 

Contact your HP representative for specific product performance information. 
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